Skip to content

feat: Introduce NePS-Spaces#248

Merged
nastaran78 merged 196 commits intomasterfrom
neps-spaces
Dec 15, 2025
Merged

feat: Introduce NePS-Spaces#248
nastaran78 merged 196 commits intomasterfrom
neps-spaces

Conversation

@Meganton
Copy link
Collaborator

@Meganton Meganton commented Oct 23, 2025

The main change is moving from dictionary-based search space definitions to using classes that inherit from neps.PipelineSpace.
This pull request updates the documentation to standardize and modernize how search spaces are defined and referenced in NePS.

Migration to class-based search space definitions:

  • The new spaces include HPO but also general CFG-like sampling capabilities. See the new docs for more information.
  • Updated all examples in docs/getting_started.md, docs/index.md, and docs/reference/neps_run.md to use a class inheriting from neps.PipelineSpace instead of a dictionary for defining the search space. This includes updating parameter definitions to use neps.Float, neps.Integer, and neps.Categorical, etc. [1] [2] [3]

Reference and terminology updates:

  • Updated method calls to use the new class-based search space, e.g., neps.run(evaluate_pipeline, ExampleSpace()) instead of passing a dictionary. [1] [2] [3]

Major formatting and clarity improvements, e.g.:

  • Clarified argument names and usage in code samples to align with the updated API. [1] [2] [3]

And so much more

lumib and others added 30 commits April 14, 2025 18:44
- Introduced tests for various aspects of the NEPS space, including:
  - `test_search_space__nos_like.py`: Tests for the resolution of a complex pipeline with unary, binary, and ternary operations.
  - `test_search_space__recursion.py`: Validates recursive operations within the pipeline, ensuring correct factor propagation.
  - `test_search_space__resampled.py`: Tests for resampling behavior in both float and integer pipelines, ensuring shared and unique values are handled correctly.
  - `test_search_space__reuse_arch_elements.py`: Verifies the reuse of architectural elements in pipelines, ensuring correct operation and configuration string generation.
  - `utils.py`: Added utility functions to generate possible configuration strings for pipelines, aiding in testing and validation.
…odule

- Changed all instances of `space` to `neps_space` in test files to reflect the new import structure.
- Updated class definitions to inherit from `neps_space.Pipeline` instead of `space.Pipeline`.
- Modified the creation of operations and categorical choices to use `neps_space.Operation` and `neps_space.Categorical`.
- Adjusted the resolution and conversion functions to utilize the new `neps_space` methods.
- Ensured consistency across all test cases for the new structure and functionality.
- Added a new module `bracket_optimizer.py` that implements a bracket-based optimization strategy for multi-fidelity configurations in NePS spaces.
- Introduced the `_BracketOptimizer` class to handle the optimization process, including sampling configurations from a prior band and managing different fidelity levels.
- Implemented various bracket types such as successive halving, hyperband, ASHA, and async hyperband.
- Updated `priorband.py` to utilize the new sampling strategies and refactored the prior sampling logic.
- Created a new `sampling.py` module to define various samplers, including `OnlyPredefinedValuesSampler`, `RandomSampler`, `PriorOrFallbackSampler`, and mutation-based samplers.
- Modified existing tests to accommodate the new optimizer and sampling strategies, ensuring compatibility with the updated structure.
- Refactored import paths in test files to align with the new module organization.
- Updated import statements to use `neps.space.neps_spaces.parameters` for all relevant classes and functions in test files.
- Refactored test classes and methods to replace direct usage of `neps_space` with the new parameters module, ensuring consistency across the test suite.
- Adjusted the instantiation of various operations and categorical choices to align with the new structure.
- Ensured that all references to operations, floats, integers, and categorical choices are correctly sourced from the updated parameters module.
- Updated imports in test files to directly import necessary classes from neps.space.neps_spaces.parameters.
- Refactored class definitions to inherit directly from Pipeline and other relevant classes.
- Replaced instances of parameter class references with direct class references for improved readability.
- Ensured consistency across test files by applying similar refactoring patterns.
- Updated imports for optimizers in test_neps_integration.py to use the new neps.optimizers module.
- Replaced references to old RandomSearch and ComplexRandomSearch algorithms with their new counterparts from neps_algorithms.
- Adjusted the optimizer imports in test_neps_integration_priorband__max_cost.py and test_neps_integration_priorband__max_evals.py to align with the new structure.
- Ensured all instances of priorband optimizers are updated to use the new neps_algorithms module.
…eration classes to improve clarity and consistency
…arity and consistency

- Updated docstrings in SamplingResolutionContext to use Args and Returns sections for better readability.
- Changed parameter names from `prior_index` to `prior` in Categorical class to enhance clarity.
- Introduced a new function `convert_confidence_level` to handle string to ConfidenceLevel conversion.
- Updated tests to reflect changes in parameter names from `prior_index` to `prior`.
- Enhanced docstrings across various classes and methods in the sampling module for consistency.
…ing clarity, and improving formatting in code examples
… HPO classes

- Updated the image segmentation example to utilize HPO_Float and HPO_Integer for hyperparameter definitions.
- Refactored test cases to replace old parameter classes with HPO variants, ensuring compatibility with the new hyperparameter optimization framework.
- Added log sampling for NePS-Integer
- Adjusted imports in test files to reflect changes in the neps library structure.
- Enhanced the configuration encoder tests to validate the new HPO parameter types.
- Ensured all relevant tests are passing with the updated hyperparameter definitions.
…md file

- Corrected links in getting_started.md, evaluate_pipeline.md, and neps_run.md to point to neps_spaces.md.
- Updated navigation in mkdocs.yml to reflect the removal of pipeline_space.md.
- Enhanced clarity in algorithms.py and grid.py by updating parameter references.
- Added validation in parameters.py to ensure prior_confidence is set when prior is defined.
…across optimizers and related functions

feat: Enhance budget logging in DefaultWorker to provide detailed resource usage status
…n normalization and logging in NEPS optimizers
@Meganton Meganton marked this pull request as ready for review December 14, 2025 21:24
@Meganton
Copy link
Collaborator Author

There is a massive backlog of parallel structures, backward-compatibility etc. in here. But we need to start somewhere, so we do it here.
See #259 for more ToDos.

@Meganton Meganton added this to the Search Space milestone Dec 14, 2025
@Meganton Meganton added documentation Improvements or additions to documentation enhancement New feature or request dependencies Pull requests that update a dependency file ci Something related to the CI system ux Something that would improve user experience labels Dec 14, 2025
@nastaran78 nastaran78 merged commit cdfab0d into master Dec 15, 2025
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci Something related to the CI system dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation enhancement New feature or request ux Something that would improve user experience

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants